Machine Learning Feature Selection Research Articles

CBDA: Chaos-based binary dragonfly algorithm for evolutionary feature selection

The goal of feature selection in machine learning is to simultaneously maintain more classification accuracy, while reducing lager amount of attributes. In this paper, we firstly design a fitness function that achieves both objectives jointly. Then we come up with a chaos-based binary dragonfly algorithm (CBDA) that incorporates several improvements over the conventional dragonfly algorithm (DA) for developing a wrapper-based feature selection method to solve the fitness function. Specifically, the CBDA innovatively introduces three improved factors, namely the chaotic map, evolutionary population dynamics (EPD) mechanism, and binarization strategy on the basis of conventional DA to balance the exploitation and exploration capabilities of the algorithm and make it more suitable to handle the formulated problem. We conduct experiments on 24 well-known data sets from the UCI repository with three ablated versions of CBDA targeting different components of the algorithm in order to explain their contributions in CBDA and also with five established comparative algorithms in terms of fitness value, classification accuracy, CPU running time, and number of selected features. The results show that the proposed CBDA has remarkable advantages in most of the tested data sets.

Trend-attribute forecasting of hourly PM2.5 trends in fifteen cities of Central England applying optimized machine learning feature selection

Recorded particulate matter (PM2.5) hourly trends are compared for fifteen urban recording sites distributed across central England for the period 2018 to 2022. They include 10 urban-background and five urban-traffic (roadside) sites with some located within the same urban area. The sites all show consistent background and peak distributions with mean annual values and standard deviations higher for 2018 and 2019 than for 2020 to 2022. The objective of this study is to demonstrate that trend attributes extracted from hourly recorded univariate PM2.5 trends at these sites can be used to provide reliable short-term hourly predictions and provide valuable insight into the regional variations in the recorded trends. Fifteen trend attributes extracted from the prior 12 h (t-1 to t-12) of recorded PM2.5 data were compiled and used as input to four supervised machine learning models (SML) to forecast PM2.5 concentrations up to 13 h ahead (t0 to t+12). All recording sites delivered forecasts with similar ranges of error levels for specific hours ahead which are consistent with their PM2.5 recorded ranges. Forecasting results for four representative sites are presented in detail using models trained and cross-validated with 2020 and 2021 hourly data to forecast 2021 and 2022 hourly data, respectively. A novel optimized feature selection procedure using a suite of five optimizers is used to improve the efficiency of the forecasting models. The LASSO and support vector regression models generate the best and most generalizable hourly PM2.5 forecasts from trained and validated SML models with mean average error (MAE) of between ∼1 and ∼3 μg/m3 for t0 to t+3 h ahead. A novel overfitting indicator, exploiting the cross-validation mean values, demonstrates that these two models are not affected by overfitting. Forecasts for t+6 to t+12 h forward generate higher MAE values between ∼3 and ∼4 μg/m3 due to their tendency to underestimate some of the extreme PM2.5 peaks. These findings indicate that further model refinements are required to generate more reliable short-term predictions for the t+6 to t+24 h ahead.

Identifying optimal variables for machine learning based fish distribution modeling

Machine learning occupies a central position in the modeling of fish distribution patterns. The augmentation of explanatory variables in fish habitat through many kinds of observational methodologies necessitates the discernment of an optimal combination of these variables for fish distribution modeling. We proposed a feature selection technique, Recursive Feature Elimination with Cross-Validation (RFECV), to determine optimal variables combinations for yellowfin tuna distribution in the Pacific Ocean. Four tree-based models, Random Forest (RF), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), and Categorical Boosting (Catboost) driven by RFECV were developed using comprehensive fisheries and biotic/abiotic data. Habitat variables including sea temperature (Temp), dissolved oxygen concentration (O2), Chlorophyll-a concentration (Chl), sea salinity (Sali), and sea surface height (SSH) were identified as significant features by all models. These models were trained based on corresponding selected variables were used to predict yellowfin tuna spatiotemporal distribution during 1995 – 2019. The results obtained could inform useful knowledge for the sustainable exploitation of yellowfin tuna in the Pacific Ocean and furnish a benchmark of feature selection for machine learning based distribution modeling of other pelagic species.

Enhancing Classification Performance through FeatureBoostThyro: A Comparative Study of Machine Learning Algorithms and Feature Selection

Early-stage prediction of a disease is an important and challenging task. The application of machine learning techniques is playing an important role in this era. Thyroid is one of the chronic endocrine diseases, and approximately 42 million people in India are affected by this disease. This paper presents a comprehensive investigation into the enhancement of classification performance through the novel ‘FeatureBoostThyro’ (FBT) model. The study evaluates various machine learning algorithms, including stochastic gradient descent (SGD), K nearest neighbor (KNN), logistic regression (LR), naive bayes (NB), and support vector machine (SVM), in conjunction with diverse feature selection methods. The research systematically explores the impact of feature selection techniques such as information gain, relief F, chi-square, gini index, forward selection, backward selection, recursive feature elimination, and LASSO on model performance across the chosen algorithms. The analysis reveals notable variations in performance metrics, including accuracy, precision, recall, and F1-score, providing valuable insights into the interplay between algorithm and feature selection. One main contribution of this research is the introduction of the FBT model, which consistently outperforms other models across various feature selection methods, making it a promising tool for addressing complex classification tasks. The findings contribute to a broader understanding of model selection and optimization in machine learning applications. The proposed model undergoes evaluation using two distinct datasets: the primary dataset acquired from Lata Mangeshkar Hospital in Nagpur and the secondary dataset obtained from the UCI dataset.

Optimizing IoT intrusion detection system: feature selection versus feature extraction in machine learning

Internet of Things (IoT) devices are widely used but also vulnerable to cyberattacks that can cause security issues. To protect against this, machine learning approaches have been developed for network intrusion detection in IoT. These often use feature reduction techniques like feature selection or extraction before feeding data to models. This helps make detection efficient for real-time needs. This paper thoroughly compares feature extraction and selection for IoT network intrusion detection in machine learning-based attack classification framework. It looks at performance metrics like accuracy, f1-score, and runtime, etc. on the heterogenous IoT dataset named Network TON-IoT using binary and multiclass classification. Overall, feature extraction gives better detection performance than feature selection as the number of features is small. Moreover, extraction shows less feature reduction compared with that of selection, and is less sensitive to changes in the number of features. However, feature selection achieves less model training and inference time compared with its counterpart. Also, more space to improve the accuracy for selection than extraction when the number of features changes. This holds for both binary and multiclass classification. The study provides guidelines for selecting appropriate intrusion detection methods for particular scenarios. Before, the TON-IoT heterogeneous IoT dataset comparison and recommendations were overlooked. Overall, the research presents a thorough comparison of feature reduction techniques for machine learning-driven intrusion detection in IoT networks.

Improving the prediction of social media engagement in universities by utilizing feature selection in machine learning

This study aims to examine the importance of feature selection in machine learning, specifically in predicting user engagement with social media post photographs on university Facebook pages. The paper uses a thorough analysis to demonstrate the crucial significance of choosing suitable features and their corresponding algorithms. The research intends to demonstrate how this strategic approach affects the accuracy of prediction findings in social media interaction. The research presents a compelling case study involving 24 leading universities from Australia, the United Kingdom, and the United States. The results underscore the efficacy of the method, stressing that the meticulous selection of characteristics and the use of appropriate algorithms are crucial elements for attaining the best results in social media forecasts. Implications: The study's results have important consequences, particularly within the changing environment of machine learning and its use in social media. Feature selection and algorithm choice are vital for optimizing social media initiatives for institutions.

MicroRNA-guided drug discovery for mitigating persistent pulmonary complications in critical COVID-19 survivors: A longitudinal pilot study.

The post-acute sequelae of SARS-CoV-2 infection pose a significant global challenge, with nearly 50% of critical COVID-19 survivors manifesting persistent lung abnormalities. The lack of understanding about the molecular mechanisms and effective treatments hampers their management. Here, we employed microRNA (miRNA) profiling to decipher the systemic molecular underpinnings of the persistent pulmonary complications. We conducted a longitudinal investigation including 119 critical COVID-19 survivors. A comprehensive pulmonary evaluation was performed in the short-term (median = 94.0days after hospital discharge) and long-term (median = 358 days after hospital discharge). Plasma miRNAs were quantified at the short-term evaluation using the gold-standard technique, RT-qPCR. The analyses combined machine learning feature selection techniques with bioinformatic investigations. Two additional datasets were incorporated for validation. In the short-term, 84% of the survivors exhibited impaired lung diffusion (DLCO < 80% of predicted). One year post-discharge, 54.4% of this patient subgroup still presented abnormal DLCO . Four feature selection methods identified two specific miRNAs, miR-9-5p and miR-486-5p, linked to persistent lung dysfunction. The downstream experimentally validated targetome included 1473 genes, with heterogeneous enriched pathways associated with inflammation, angiogenesis and cell senescence. Validation studies using RNA-sequencing and proteomic datasets emphasized the pivotal roles of cell migration and tissue repair in persistent lung dysfunction. The repositioning potential of the miRNA targets was limited. Our study reveals early mechanistic pathways contributing to persistent lung dysfunction in critical COVID-19 survivors, offering a promising approach for the development of targeted disease-modifying agents.

Quantum computer based feature selection in machine learning

AbstractThe problem of selecting an appropriate number of features in supervised learning problems is investigated. Starting with common methods in machine learning, the feature selection task is treated as a quadratic unconstrained optimisation problem (QUBO), which can be tackled with classical numerical methods as well as within a quantum computing framework. The different results in small problem instances are compared. According to the results of the authors’ study, whether the QUBO method outperforms other feature selection methods depends on the data set. In an extension to a larger data set with 27 features, the authors compare the convergence behaviour of the QUBO methods via quantum computing with classical stochastic optimisation methods. Due to persisting error rates, the classical stochastic optimisation methods are still superior.

Multi-institutional prognostic modeling of survival outcomes in NSCLC patients treated with first-line immunotherapy using radiomics

BackgroundImmune checkpoint inhibitors (ICIs) have emerged as one of the most promising first-line therapeutics in the management of non-small cell lung cancer (NSCLC). However, only a subset of these patients responds to ICIs, highlighting the clinical need to develop better predictive and prognostic biomarkers. This study will leverage pre-treatment imaging profiles to develop survival risk models for NSCLC patients treated with first-line immunotherapy.MethodsAdvanced NSCLC patients (n = 149) were retrospectively identified from two institutions who were treated with first-line ICIs. Radiomics features extracted from pretreatment imaging scans were used to build the predictive models for progression-free survival (PFS) and overall survival (OS). A compendium of five feature selection methods and seven machine learning approaches were utilized to build the survival risk models. The concordance index (C-index) was used to evaluate model performance.ResultsFrom our results, we found several combinations of machine learning algorithms and feature selection methods to achieve similar performance. K-nearest neighbourhood (KNN) with ReliefF (RL) feature selection was the best-performing model to predict PFS (C-index = 0.61 and 0.604 in discovery and validation cohorts), while XGBoost with Mutual Information (MI) feature selection was the best-performing model for OS (C-index = 0.7 and 0.655 in discovery and validation cohorts).ConclusionThe results of this study highlight the importance of implementing an appropriate feature selection method coupled with a machine learning strategy to develop robust survival models. With further validation of these models on external cohorts when available, this can have the potential to improve clinical decisions by systematically analyzing routine medical images.

Improved prediction of settling behavior of solid particles through machine learning analysis of experimental retention time data

The motion of particles through density-stratified interfaces is a common phenomenon in environmental and engineering applications. However, the mechanics of particle-stratification interactions in various combinations of particle and fluid properties are not well understood. This study presents a novel machine-learning (ML) approach to experimental data of inertial particles crossing a density-stratified interface. A simplified particle settling experiment was conducted to obtain a large number of particles and expand the parameter range. Using ML, the study explores new correlations that collapse the data gathered in this and in previous work by Verso et al. (2019). The “delay time”, which is the time between the particle exiting the interfacial layer and reaching a steady-state velocity, is found to strongly depend on six dimensionless parameters formulated by ML feature selection. The data shows a correlation between the Reynolds and Froude numbers within the range of the experiments, and the best symbolic regression is based on the Froude number only. This experiment provides valuable insights into the behavior of inertial particles in stratified layers and highlights opportunities for future improvement in predicting their motion.

Overview of the whalse algorithm for feature selection and extraction in machine learning

Feature selection and extraction play a key role in machine learning, helping to improve model performance and reduce data complexity. In the context of feature selection and extraction in machine learning, the Whales approach, also known as the Whale Optimization Algorithm (WOA), is often used. The unique properties of the Whales algorithm and its optimization capabilities can be useful in solving problems of feature selection and extraction. This article contains both a review of similar methods and an analysis of the Whales algorithm itself and the main stages of its application. The paper also examines the relationship between the Whales algorithm and traditional methods of feature selection and extraction and identifies future directions for research.

Weighted Feature Selection for Machine Learning Based Accurate Intrusion Detection in Communication Networks

Weighted Feature Selection for Machine Learning Based Accurate Intrusion Detection in Communication Networks

A Multiform Optimization Framework for Multi-objective Feature Selection in Classification

Feature selection in machine learning as a key data processing technique has two conflicting goals: minimizing the classification error rate and minimizing the number of features selected. However, most of the existing multi-objective feature selection methods face the problems of easily falling into local optima and slow convergence by virtue of their problem characteristics, such as partially conflicting objectives and highly discontinuous Pareto fronts. To solve these problems, this article proposes a multiform optimization framework to solve a multi-objective feature selection task together with several auxiliary single-objective feature selection tasks in a multitask environment. The proposed framework uses the problem-solving experience of single-objective tasks to assist the multi-objective feature selection task in exploring more promising regions and accelerating the convergence speed. Specifically, a knowledge transfer strategy based on the search experience of different tasks is developed to accomplish multiform optimization. In addition, a diversity enhancement mechanism is presented to improve search ability in promising decision space areas by considering historical information about the population. In most cases, the experiment results on 27 datasets demonstrate that the proposed technique can uncover more diversified feature subsets on the Pareto front in less time than existing state-of-the-art methods.

An Extensive Analysis of Feature Selection in Machine Learning

Abstract: Researchers and engineers working in the disciplines of data mining and machine learning have difficulties while analysing high-dimensional data. A dimension reduction method called feature selection is used to pick characteristics that are pertinent to machine learning tasks. Improving the efficiency of machine learning algorithms, hastening the learning process, and creating basic models all depend critically on reducing the size of the dataset by removing superfluous and useless information. Numerous feature selection techniques have been put forth in the literature to find the pertinent feature or feature subsets in order to accomplish the goals of clustering and classification. The purpose of this study is to review the state of the art for these methods.

Applying sustainable development goals in financial forecasting using machine learning techniques

AbstractThis study seeks to identify the impact of sustainable development goals (SDGs) in predicting corporate financial performance (CFP) in the information communications technology (ICT) industry. Data over the period of 2016–2020 that are relevant to financial reporting and corporate social responsibility (CSR) reporting have been extracted for 208 firms in the ICT industry. Important variables have been identified to help predict the financial performance in the following years upon the publication of CSR reports. Drawing on resource‐based view and stakeholder theory, the purpose of this study is to find the quintessential variables that influence the prediction accuracy of financial performance. To better forecast earnings per share (EPS), machine learning feature selection methods have been implemented. The findings suggest that certain variables such as return on total assets, SDGs adoption and whether the firm has established KPI for SDGs achievements can help enhance EPS prediction. With the various predictive models, the artificial neural network model is the most effective in predicting CFPs. Most importantly, the adoption of SDGs can be utilized to sharpen the forecast on financial performance as it enables firms to bolster stakeholder engagement and evaluate environmental, social, and corporate governance efforts.

Utilizing deep learning, feature ranking, and selection strategies to classify diverse information technology ticketing data effectively

In today's internet world, information technology (IT) ticketing services are potentially increasing across many corporations. Therefore, the automatic classification of IT tickets becomes a significant challenge. Feature selection becomes most important, particularly in data sets with several variables and features. However, enhance classification's precision and performance by stopping insignificant variables. Through our earlier research, we have categorized the unsupervised ticket dataset. As a result, we have converted the dataset into a supervised dataset. In this article, the classification of different IT tickets on Machine learning algorithms, Feature ranking, and feature selection techniques are used to improve the efficiency of machine learning algorithms. However, compared to the machine learning (ML) algorithms, the convolutional neural network (CNN) algorithm provides a better classification of the token IDs and provide better accuracy.

Application of machine learning algorithms and feature selection methods for better prediction of sludge production in a real advanced biological wastewater treatment plant

Although the management of sewage sludge is an important and challenging task of wastewater treatment, there is a scarcity of studies on the prediction of waste sludge. To overcome this deficiency, the present work aims to develop an appropriate model providing accurate and fast prediction of sewage sludge. With this aim, different machine learning (ML) algorithms were tested by data obtained from a real advanced biological wastewater treatment plant located in Kocaeli, Turkey. In modelling studies, a data set from January 2022 to December 2022 composed of 208 daily measurements was considered. The flow rate of the plant (Q), polyelectrolyte dosage (PD) and removed amounts of total suspended solids (TSS), chemical oxygen demand (COD), biological oxygen demand (BOD), total phosphorous (TP), total nitrogen (TN) were assigned as input parameters to predict sludge production (SP). The precision of the models was evaluated in terms of Mean Square Error (MSE), Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and correlation coefficient (R2). Among the various tested models Kernel Ridge Regression provided the best accuracy with R2 value of 0.94 and MAE value of 3.25. Mutual information-based feature selection (MIFS) and correlation-based feature selection (CFS) algorithms were also used in the study in order to enhance the model performance. Thus, higher prediction accuracies were achieved using the selected subset of features. Furthermore, importance contribution of features were calculated and visualized by SHapley Additive exPlanations (SHAP) technique. The overall results of the work indicate the feasibility of ML models for describing the dynamic and complex nature of SP. The process operators may benefit from this modelling approach since it enables accurate and fast estimation of sewage sludge by using fewer and easily measurable parameters.

Integrated framework for hydrologic modelling in data-sparse watersheds and climate change impact on projected green and blue water sustainability

Climate and hydrologic hazards pose a threat to the distribution of watersheds’ water resources in time and space, necessitating planning for sustainable resilience and adaptation. Hydrologic modelling has emerged as a potential solution for understanding watershed responses to projected climate change, and a prediction model that can deliver actionable information is necessary, although it requires basin-scale observations to calibrate the model to reliably predict basin-scale water resources hazards. Such luxury is not always tenable in watersheds with inadequate ground-based observation. However, satellite-based evapotranspiration (ET) data coupled with a machine learning feature selection as a data refinement process has made integrated water balance modelling widely regarded as a viable alternative for improving the capability of watershed modelling processes in data-sparse regions. This study developed a convincing hydrologic model framework to sufficiently calibrate and provide accurate behavioural solutions for all model responses. The framework was applied to four sub-basins that form the larger Lake Chad basin. The model results were applied to assess the dynamic changes in projected blue and green water resource sustainability in response to climate change in one of the sub-basins. Study findings indicate that hydrologic fluxes can be simulated accurately with varying degrees of acceptability, with R2 and NSE values in the range of 0.69–0.88 and 0.45–0.77 for calibration and 0.69–0.79 and 0.34–0.63 for validation, respectively, and captured within a satisfactory uncertainty range of P-factor and R-factor values of 0.68–0.93 and 0.73–1.31, respectively, in 83%, 67%, 85.7%, and 81.3% of the sub-watersheds based on multi-site simulation despite distinct watershed morphology, although there are significant trade-offs in parameter sensitivity. Whilst green water is the dominant freshwater component across the basin relative to blue water, climate change may be a significant factor influencing changes in the projected green water sustainability status, and the combination of socioeconomic drivers and climate change may significantly impact the projected blue water sustainability status across the basin. Projected changes in the green and blue water sustainability status have shown that more than 50% of the watershed will become ecologically fragile. In addition, the identified freshwater geographic sustainability hotspots may be beyond restoration without adequate long-term river basin water resource plans.

A hybrid machine learning feature selection model-HMLFSM to enhance gene classification applied to multiple colon cancers dataset.

Colon cancer is a significant global health problem, and early detection is critical for improving survival rates. Traditional detection methods, such as colonoscopies, can be invasive and uncomfortable for patients. Machine Learning (ML) algorithms have emerged as a promising approach for non-invasive colon cancer classification using genetic data or patient demographics and medical history. One approach is to use ML to analyse genetic data, or patient demographics and medical history, to predict the likelihood of colon cancer. However, due to the challenges imposed by variable gene expression and the high dimensionality of cancer-related datasets, traditional transductive ML applications have limited accuracy and risk overfitting. In this paper, we propose a new hybrid feature selection model called HMLFSM-Hybrid Machine Learning Feature Selection Model to improve colon cancer gene classification. We developed a multifilter hybrid model including a two-phase feature selection approach, combining Information Gain (IG) and Genetic Algorithms (GA), and minimum Redundancy Maximum Relevance (mRMR) coupling with Particle Swarm Optimization (PSO). We critically tested our model on three colon cancer genetic datasets and found that the new framework outperformed other models with significant accuracy improvements (95%, ~97%, and ~94% accuracies for datasets 1, 2, and 3 respectively). The results show that our approach improves the classification accuracy of colon cancer detection by highlighting important and relevant genes, eliminating irrelevant ones, and revealing the genes that have a direct influence on the classification process. For colon cancer gene analysis, and along with our experiments and literature review, we found that selective input feature extraction prior to feature selection is essential for improving predictive performance.

Heart disease diagnosis Performance evaluation of supervised machine learning and feature selection techniques

Heart disease diagnosis is a critical task in modern healthcare, demanding accurate and efficient methods. This study focuses on evaluating the performance of various supervised machine learning algorithms and feature selection techniques for the diagnosis of heart disease. The algorithms considered include Naïve Bayes, Decision Tree, KNN, SVM, and Logistic Regression. Additionally, different feature selection techniques are employed to identify the most relevant features for improved diagnostic accuracy. Furthermore, the study investigates the impact of feature selection on algorithm performance. Feature selection techniques are applied to identify the subset of attributes that contribute most effectively to heart disease diagnosis. The combination of various algorithms and feature selection methods yields insights into which approaches are most suitable for accurate and efficient heart disease diagnosis